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Abstract. We study Schur Q-polynomials evaluated on a geometric progression, or equi- 
valently q-enumeration of marked shifted tableaux, seeking explicit formulas that remain 
regular at q = 1. We obtain several such expressions as multiple basic hypergeometric se- 
ries, and as determinants and pfaffians of continuous q-ultraspherical or continuous q-Jacobi 
polynomials. As special cases, we obtain simple closed formulas for staircase-type partitions. 



1. Introduction 

The Schur Q-polynomials originate in the work of Schur jSc] on projective rep- 
resentations of the symmetric group. Nowadays, they are recognized as the case 
t — — 1 of the Hall-Littlewood polynomials, which are in turn a special case of the 
Macdonald polynomials jMaj . 

In the present paper we report on some investigations on Schur Q-polynomials 
that were motivated by applications to sums of squares |R3j . In |Rlj . we used 
elliptic pfaffian evaluations to derive, and generalize, certain triangular number 
identities conjectured by Kac and Wakimoto |KWj and proved first by Milne 
|M2| IM31 IM4j and later by Zagier [Zj . Extending this work to the case of square 
numbers leads naturally to quantities related to Schur Q-polynomials. To be 
precise, we note that, although one usually assumes that the polynomial Q\ is 
indexed by a partition A, the definition (|2.1|) makes sense for general A e Z m . The 
quantities alluded to may then be written Q( A _ A )(1, . . . , 1), where A is a partition. 
In the present paper, however, we investigate Q\(l n ) for general A. 

In many ways, Schur Q-polynomials and Schur polynomials have analogous 
properties. Therefore, it may be instructive to compare with the much simpler 
situation for the latter. The standard definition 



s x lx 



deti<^< n (x 8 Aj+ " 3 ) 
deti<io< n (x"" J ) 



L<j ,j<n \ 

has a removable singularity at x% = ■ ■ ■ = x n = 1. A well-known way to compute 
s\(l n ) is by passing to the case when the variables are in geometric progression. 
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Namely, by the Vandermonde determinant evaluation, 

M n-l\ lll<i<j<nW y ) /. 

* l(1 ' 9 ""' 9 )= XL^^-t") • (L1) 

which in the limit q — > 1 gives 

Aj - A, + j - i 



7 — z 

l<i<j<n J 

Our method for computing Q\(l n ) is similar. In that case, Q\(l, q, ■ ■ ■ , <? n_1 ) is 
a multiple basic hypergeometric sum, see ()2.5|) . again with a removable singularity 
at q — 1. Since it does not simplify in general, the best one can hope for is to 
find a transformation formula, leading to a different sum without the apparent 
singularity. In the limit q — > 1 one would then obtain a formula for Q\(l n ) as a 
multiple classical hypergeometric sum. Finding such identities is our main goal. 

It should be remarked that Q\(l, q, . . . , g n_1 ) has a combinatorial meaning as 
a generating function (or g-enumeration) on marked shifted tableaux, see (|2.3j) . 
Thus, we can equivalently formulate our goal as finding explicit solutions to the 
g-enumeration problem that remain regular in the limit q — > 1, corresponding to 
classical enumeration. 

The key fact for finding transformation formulas with the desired property is 
Lemma 15.11 which allows us to identify the two-row Q-polynomial 

Q(Ai,A 2 )( 1 )9 ) ---)0 

with the Christoffel-Darboux kernel of certain continuous g-Jacobi polynomials. 
Like most of our results, it is formulated in terms of multivariable polynomials P n 
that are related to Q\ through 

Q A (l,g,...,g") = 2 m P n (g\...,g A ™). (1.2) 

The multi-row Q-polynomial similarly corresponds to a multivariable Christoffel- 
Darboux kernel. This fact implies several pfaffian and determinantal formulas 
for the polynomials P n . Besides the pfaffian formula (|2.27j) . which is essentially 
Schur's definition of the Q-polynomial, we have the transformed pfaffian formula 
of Corollary 15.71 and the determinant formula of Corollary 15.81 In the limit q — > 1 , 
the latter leads to an elegant determinant formula for Q\(l n ), see Corollary 15.101 

We then use our new pfaffian and determinantal formulas to obtain hyper- 
geometric formulas. Corollary 15.71 allows us to express P n as a pfaffian of double 
sums. Series manipulation then leads to the multiple hypergeometric sum of The- 
orem 16.11 Similarly, Corollary 15.81 expresses P n as a determinant of single sums, 
which leads to the Schlosser-type hypergeometric sums of Theorems 17. Il and l7. 31 In 
the limit q — > 1, Theorems 16. II and 17.31 yield hypergeometric formulas for Q\(l n ), 
thus fulfilling our main goal, f Theorem 17.11 retains the singularity at q = 1.) 

In some special cases, our expressions for Q\(l, q, . . . , g n_1 ) simplify, giving com- 
pletely factored expressions similarly as for Schur polynomials. 
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• n = oo, see Section 0] and Remark 15.21 The explicit evaluation of 

Qx(l,q,q 2 ,...) 

follows from an identity of Kawanaka jKj, who does not, however, mention 
the relation to Q-polynomials. 

• A staircase, see Remark 17.41 If A = (to, m — 1, . . . , 1), then Q\ = 2 m s\, so 
g A (l,g,...,g n " 1 ) factors by {TIJ). 

• A odd staircase, see Section HI Remark 16.31 and Remark 17.41 If A = 
(2m — 1, 2m — 3, . . . , 1), then Q\(l, q, . . . , g n_1 ) can be evaluated using a 
discrete Selberg integral due to Milne |Mlj . or in the form needed here to 
Krattenthaler |Krj . Again, the connection to Q-polynomials seems not to 
have been noticed before. 

• A even staircase, see Corollary 17.61 If A = (2m, 2m — 2,..., 2), then 
Qx(l, <?,•••, q n ~ l ) factors. In contrast to the previously mentioned cases, 
we have not been able to reduce this to previously known results. 

The plan of the paper is as follows. Section|2]contains preliminaries. In Sectional 
we consider the limit n — > oo and in Section 0] the odd staircase, describing the 
connection to previously known results. After this preliminary material, we lay the 
foundation for our main results in Theorem 15. 41 where we relate Q\(l, q, ■ ■ ■ , q n ) to 
multivariable Christoffel-Darboux kernels, obtaining as a consequence new pfaffian 
and determinantal identities. These are then used to derive the pfaffian hyper- 
geometric identity of Theorem 16 . 1 1 and the determinantal hypergeometric identities 
in Section [7| Finally, in the Appendix we give an alternative proof of Theorem 15. 41 

Acknowledgement: I thank John Stembridge for several useful comments. 

2. Preliminaries 

We will use the terminology and notation of [Ma] for partitions and symmetric 
functions, and that of |GRj for classical and basic hypergeometric functions. 

2.1. Schur Q-polynomials and marked shifted tableaux. When m < n and 
A = (Ai, . . . , A m ) is a partition of length m, that is, 

Ai > A 2 > • ■ ■ > A m > 0, 

the Schur Q -polynomial Q\ is defined by 




l<i<j<n 



Here, S n acts by permuting the variables Xi,...,x n and S n _ m is the subgroup 
acting on x m+ %, . . . ,x n . Note that Q\ = unless A is strict, that is, 

Ai > A 2 > • ■ ■ > A m > 0. 

For strict A, Q\ has a combinatorial interpretation in terms of marked shifted 
tableaux. We briefly recall the relevant definitions. 
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Let A be the diagram of a strict partition, and let S(X) denote the diagram 
obtained by shifting the i-th row (i — 1) steps to the right, for each i. A marked 
shifted tableau of shape S(X) is a labelling of the boxes of S(X) with symbols from 
the ordered alphabet V < 1 < 2' < 2 < • • • such that: 

(1) The labels increase weakly along rows and down columns. 

(2) Each unmarked symbol occurs at most once in each column. 

(3) Each marked symbol occurs at most once in each row. 

Let afc be the number of boxes labelled either k or k', and let x T denote the 
monomial n^i 3 ^- Then, |Ma| (III. 8. 16')] 

Q A (x) = ]>> T , (2.2) 

T 

where the sum is over all marked shifted tableaux of shape 5(A). In ()2.2|) the 
number of variables is infinite; restricting the alphabet to 1' < 1 < • • • < n' < n 
gives a formula for Q\(xi, . . . , x n ). 

When T is a marked shifted tableau, we let |T| = ^2 k>1 (k — 1) a k, with as 
above. Then, 

Q x (l,q,...,q n - l ) = Y,Q m (2-3) 

T 

is the generating function for |T| on marked shifted tableaux of shape A and 
alphabet V < 1 < • • • < n' < n. In particular, the cardinality of this set is Q\{l n ). 

2.2. Schur Q-polynomials and basic hypergeometric series. Representing 
a by the m-tuple (fc 1; . . . , k m ) = (cr(l), . . . , <r(m)), we may rewrite (|2.1|) as 

Qx(xi, ...,x n ) 

m m 

=2- e ik n ^fn n t^f 

l<fc lv ..,fe m <n i=l l<i<j<m k% k J i=l ie{l,...,n}\{fci,...,fc m } ki 3 
ki distinct 

n m m n 

=2-5: ik n J^fn n ^ 

fcl,...,fc m =l i=l l<i<j<m k 3 kl t=l j=l,j^ki kl 3 

Consider the case when xi,...,x n are in geometric progression. Then, (|2.4|) is a 
multiple basic hypergeometric sum. Indeed, replacing n by n + 1 and ki by fcj + 1, 
we obtain after simplification 



Qx(l,q,...,q n ) 



fa ^ fcl ,.^1=0 K,i< m ^ + ^ tl («' ^ 

a; q) k = (1 - a)(l - ag) • • • (1 - ag fc_1 ) 



ci,...,K m =0 l<i<j<m 

where we use the standard notation IGR 
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(ai, ...,a m ; q) k = g)& • • • (a m ; g)^. 

It will be convenient to view q Xi as free variables. Thus, we introduce the 
polynomials 



■fji (^"1 j • • • j "^r. 



(q;q)l 



gfcj " gfc 'nS: 9,9 -I: 9k 4% (2.6) 



r n 

fel ,:x=o i<ii<m qkj + ti ^ ~ q ~ n i *)* 



so that ()1.2|) holds. By anti-symmetry, we may equivalently write 

Pni-^l j • • • j ^m) 



^-9;g)„ 
fa;?)? 



e n 

0<fc m <---<fci<n l<i<.7<m 



1. . J.™ 



n 



-g,9 n ;?)* 



gfc, + q k t 11 fa _ g n. g ) fe 



det (40- (2-7) 



This can be viewed as a Schur polynomial expansion, see (|3.2|) . One easily verifies 
that 



P n (xi, . . . ,ac m ,0) = (-l) r 



(g; Q)n 



•^m Pn—1 j 



(2- 



For m = 1, P n is a terminating well-poised 2 </>i series, and can thus be expressed 
in terms of continuous g-ultraspherical polynomials |GR| as 



Pjx) 



(?; ?)t 



2^1 



-9,9" 



g,x 



e ine C n (cos#;-g|g), x = -e^ y . (2.9) 



2i.e 



It will be convenient to introduce the monic polynomials 

c n (x) = q,<1 " C n (x/2;-g|g). 
(-9; 9)n 

Then, assuming a fixed choice of \f—x, 

-9; 9) 



(9;9)r 



(2.10) 



(2.11) 



For < q < 1, the polynomials (cfc)^ form an orthogonal system with respect to 
a unique positive measure (see (jA.ljl ). which we can normalize so that 



\Ck\ 



(Q;Q)k(q;q) 



fc+i 



[-q;q)k(-q; q)k+i 



(2.12) 



2.3. Orthogonal polynomials. Our main interest is in the case 5 = 1. Unfortu- 
nately, the right-hand side of ()2.5|) is then singular. However, when m = 1 one can 
use transformation formulas for basic hypergeometric series to derive a plethora 
of other expressions, many of which remain regular when q — > 1. 

The transformations that we will use correspond to different identifications of 
g-ultraspherical polynomials with continuous g-Jacobi polynomials, which in turn 
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form a sub-class of the Askey-Wilson polynomials. Recall that the latter poly- 
nomials are defined by 



(ab,ac,ad;q). 
p n {cos9;a,b,c,d\q) = 4 ^3 



g n ,abcdq n 1 ,ae t6 ,ae 10 
ab, ac, ad 



a" 

they are symmetric in the parameters a, b, c, d and satisfy 

p n (-x; a, b, c, d\q) = (-l) n p n (x; -a, -b, -c, -d\q). (2.13) 
By HIE1 Eq. (7.5.33-36)], 

C n (x; a\q) = - ^ ^" p n (x; ^/a, -v 7 ^, — v/^k) (2.14a) 

W' a Q i Q)n 

p n (x ; ^,-^,g 1/4 ,-g 1/4 l? 1/2 ), (2.14b) 



(q,aq 1 / 2 ;q) n 

C 2n (x- a\q) = p^ Pn (2x 2 - 1; a, -1, y/q, -y/q\q) (2.14c) 
((r: ' r) ' 1 ■^(2x 2 -l;a,ag,-l,-g|g 2 ), (2.14d) 



C 2 n+i(^; a|g) = } Q ' ^ n+1 2xp n {2x 2 - 1; a, -q, y/q, -y/q\q) (2.14e) 
(a 2 ;g 2 ) n+1 - l;o,og, -g, -g 2 !? 2 )- (2.14f) 



(g, -a; g) 2 n+i 



It is easy to prove this by verifying that the orthogonality measures for the various 
polynomials agree. 

In view of (12. 9|) . we are particularly interested in the case a = —q. Using also 



2.13|) . we may then write (|2.14J) as 

P n {-e™) = ( ~ g ; 2 g) " e M p n (cose;iq^,-iq^,iq-iq\q) (2.15a) 
(g,g n +^;g) n 



n And 



(g, -g 3 / 2 ;g)„ 



p n (cos 6- iq l '\ -iq x l\ q x '\ -g 1/4 |g 1/2 ), (2.15b) 



P 2 n(x) = X > n (I( X + x-l); 1, g, ^g, _^g|g) ( 2 .16a) 



^J. 9 \2 U x n Pn(\{x + x l ); l,g,g,g 2 |g 2 ), 

2n 



(g; g) 2 



(2.16b) 



SCHUR Q-POLYNOMIALS, HYPERGEOMETRIC SERIES AND TABLEAUX 



7 



P2n+i{x) = ^ f _ q, j^ n+1 x n 0- - x)p n {\{x + x 1 ); g, g, y/q, -y/q\q) (2.16c) 



(?;?) 



2n+l 



(g 2 ;g 2 ' 



ln+1 



(?; 9)in+l 



x n (l - x) p„(±(a; + x g, g, g 2 , g 2 |g 2 ). 



(2.16d) 



These identities yield many useful expressions for P n (x). For instance, (|2.15a|) 
implies 



Pn(x) 



1-g 



n+l 



1-g 



a/^A/) 403 



g n , g n+2 , \Jq/x, —^fqx. 



g,g 3 / 2 ,-g 3 / 2 ' g ' 9 



(2.17) 



In contrast to (|2.9|h this expression is regular in the limit 



Q (A) (l n+1 ) = lim2P n (g A ) = (2n + 2) 3 F 2 



-n,n + 2,(l-A)/2 
1,3/2 



(2.18) 



This will be generalized to general m in Theorem 16.11 and Corollary 16.21 
Similarly, (|2.15b|) gives 



Pn(x) 



1-g 



n+l 



1-g 



V 7 ^/?) 403 



r n/2 ) _ ? (n+2)/2 ) y^ ) _^ 



g, zg 3//4 , — ig 3 / 4 



g ' ,g 



which implies the simple formula 



Q(A)(1 



lim2P n (g ; 

g-t-l 



(2n + 2) 2 F 1 



—n, 1 — A 



;2 



(2.19) 



In Section we will need the following eight expressions, which are all conse- 
quences of f!2.16|) : 



P2n(x) 



1-g 



2n+l 



1-g 

(g 3/2 ,-g; g) 
(g, -gV 2 ;g) 

1 — g2n+l 
1-? 



g- n x%03 



g n , — g n+1 , gx, g/x 
g,g 3 / 2 ,-g 3/2 ' ^ ? 



g 2 a; 403 



g n , — g n+1 , g 1//2 x, g 1 / 2 /^ 
qV 2 ,q 3 / 2 ,-q 5 



9,9 



g~ n a; n 403 



g 2n ,g 2n+2 ,gx,g/x 2 2 
g, g 2 , g 3 ' « ' « 



1-g 



2n+l\ 2 



1-g 



g- 2 "x%0 3 



g 2n ,q 2n+2 ,q 2 x,q 2 /x 2 2 



(2.20a) 
(2.20b) 
(2.20c) 
(2.20d) 
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^2n+l[X) — ^ _ ^ 2 q X 4 </) 3 

(q 3/2 ;q)n(-q;q)n+i(i- x) 



(i - q)(q,-q 3/2 ;q)n 

;i - q 2n+2 )(l - x) 



q 2 X '403 



q n , —q n+2 , qx, q/x 
g 2 ,g 3 / 2 ,-g 3 / 2 ; q ' 

< n+2 qV^qW/x 



q 3 / 2 , g 3 / 2 , — q 



q,q 



(l-g) 2 
[l-q 2n+2 ) 2 (l-x 

' (l-g) 2 (l-g 2 ) 



q- n x n ,h 



q 2n ,q 2n+4 ,qx,q/x 2 2 

q ■> q i q 



-2n n l 
q X 4(£>3 



9 2n ,q 2n+ \q 2 x,q 2 /x 2 2 

q i q i q 



(2.20e) 
(2.20f) 
(2.20g) 
(2.20h) 



In the limit g — > 1, the Askey-Wilson polynomials in ()2.15|) and ()2.16|) degener- 
ate to four different systems in the Askey Scheme. Let us elaborate on this point, 
introducing the notation 



Pk(x) 



i k k\ 



limP fc (g" 



2 k 9-1 

where the constant is chosen so as to make pk monic. By ()2.I8)1 and ([2.19J1 . 



, x i k (k + l)\ „ 



—k, 1 + ix 



;2 



i k {k + 1)! 



3-^2 



-n,n + 2,(l+ix)/2 
1,3/2 ' 



It follows that pk may be identified with Meixner-Pollaczek polynomials and with 
continuous Hahn polynomials. Explicitly, in the notation of |KSj . 



p k (x) = ^ k P k [1 \x;n/2) 



2 k k\(k + l)\ 
(2fc + l)! 



P k (x/2; 1/2, 1,1/2,1). 



In particular, (pk{ x ))kLo are the mon ic orthogonal polynomials corresponding to 
the measure 

X 

— T7 — r /O) da;, 
.oo smh(7rxj 



with norms 



Let 



\\Pk\[' 



fc!(fc + l)! 

22fe+l ' 



P2fc(a;) = pi 0) (^ 2 ), P2k+i(x) = xp[ l \x 2 ), 



(2.21) 



so that {p k £ \x))% = 

o are the monic orthogonal polynomials corresponding to 



x £ f(x) 
sinh(7r v /x) 



£ = 0, 1, 



with norms 



(0) 2 _ (2fc)!(2fc + l)! 



24fc+l 



(1) 2 _ (2A; + l)!(2fc + 2)! 



24fc+3 



(2.22) 
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f) 



Then, are continuous dual Hahn and Wilson polynomials; explicitly, 

pf\x) = (-l) k S k (x; 0, 1/2, 1) = W k (x/A- 0, 1/2, 1/2, 1), 

P k 1] (x) = (-l) k S k (x; 1/2, 1, 1) = (-l) fe ^^r W k (x/A; 1/2, 1/2, 1, 1). 

This may be seen by letting q — > 1 in (|2.2(J|) . or by checking that the orthogonality 
measures agree. 

Remark 2.1. It may seem strange that the variables Aj, which are originally 
integers, have to be taken as imaginary at the support of the orthogonality measure 
of p k - Actually, if we let 

-k, 1 — x 



_ pk(jx) _ (fc+ 1)! 



(2.23) 

then f k are "almost" orthogonal on the positive integers, in view of the Abel sum 
gjf(-l)^/ m (fc)/ n (fc) = ( ~ ir ^ + t 8™. 

k=l 

This type of orthogonality plays an important role in |R3j . It is a limit case of 
the orthogonality of Meixner polynomials M n (x;/3,c) jKSj with (3 = 2, c — > — 1. 
When g / 1, a similar formula can be obtained as a limit case of the discrete 
orthogonality for g-ultraspherical polynomials considered in |AK| . 



2.4. Schur Q-polynomials and pfaffians. The pfaffian of a skew-symmetric 
even-dimensional matrix is defined by 

m 

pfaff {a ij )= ^ s 8 n ( a ) Y[ a *(2i-i),cT(2i), 

Ki.j<2m _ . . 

- J ~ <r&S 2m /G i=l 

where G is the subgroup of order 2 m m! consisting of permutations preserving the 
set of pairs {{1, 2}, {3, 4}, . . . , {2m — 1, 2m}}. Pfaffians and Schur Q-polynomials 
are intimately related. Indeed, Schur's original definition of the latter jSc] is based 
on the identity 

Q(Ai,...,A 2 m) — Pfaff {Q(Xi,x 3 )) ■ (2.24) 

l<M<2m 

In the same work, Schur obtained the pfaffian evaluation 

ry . ry m \ /y , i-v» , 



pfaff p— ^ = If ^— ii, (2.25) 

l<tj<2ro \ X i + ^i/ l<i<j<2-m ^ + X * 

which we recall here together with its companion |Stel Proposition 2.3] 

(ry ry \ ry . ry . 

3 1 =f' H "> n ] [ 3 _ 1 . (2.26) 
1 — tXiXj J Ki<j<2m ~ ^ X i X 3 
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We need an elementary property of pfaffians, which is stated below as Corol- 
lary 1231 Incidentally, it can be used to prove (|2.24|) . see Remark 12.51 Before 
formulating the result, we find it instructive to give a generalization. 

Lemma 2.2. Let fii)i<i<2m be a collection of measure spaces, and (&ij)i<ij<2m 
a collection of integrable functions, bij : x Xj — > M, such that bij(x,y) = 
—bji(y,x). Then, 



pfaff ( / / bij(x,y) dfi i (x)dfi j (y) 

l<M<2m 



pfaff (bij(xi, xj)) dni(xi) ■ ■ ■ dfj lm (x 2m ). 

l<i,j<2m 



Proof. The left-hand side equals 

S S^)fl j j ^(2i-l), CT (2i)(^,yi)^(2i-l)(^)^(2i)(2/i)- 

Introducing new integration variables by Xi t— > x a (2i-i), y% i— > £<7(2i) and interchang- 
ing the finite sum and the integral, we obtain the desired result. □ 

Consider the special case of Lemma 12.21 when each Xj is equal to the same 
finite space X. Writing X = {1, . . . , n}, fii(j) = A^, bij(k, I) = B^, it takes the 
following form. 

Corollary 2.3. Let (Aj)i<i<2m,i<j<n and (#£0 i<i,j<n, i<fc,z<2m be matrices such 
that Bf- = -B l h. Then, 

n 2m 



pfaff A ^vK) E II ^ p faff (^)- 

l<i,i<2m \ Vx y=1 y fe 1 ,...,fe 2m =l i=l ^.i^ 2 ™ 

Remark 2.4. We will only need Corollary 12.31 in the case when By = Bfj is 
independent of k and I. It can then equivalently be written 

viafi(ABA t )= V det (A^) pfaff (%.), 

which is a version of the "minor summation formula" of Ishikawa and Wakayama 
[IWl]. The corresponding special case of Lemma f2. 21 is due to de Bruijn jBj. 



Remark 2.5. To prove ()2.24j) . one may apply Corollary 12 .31 in the case when 

n 



A _ »Ai TT ^fe J" X J rijfci _ d 



/-v» <-yj , J " />i , _l_ <-yj , 



Using (|2.25jl to compute pfaff (B^.) and comparing with ()2.4j) gives indeed (|2.24j) . 
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Similarly, choosing 



jfc-i (-g»g n ;g)fc-i b h = b = qj - ql 



it follows from (|2.fij) that, for m even, 

P n (xi, ■ ■ ■ ,x m ) = pfaff (P n (xi,Xj)). (2.27) 

l<i j<m 

Together with (|2.8)L this reduces the study of the polynomials P n (xi, . . . ,x m ) to 
the case m = 2. 

2.5. Mult i variable Christoffel— Darboux kernels. Let 



f^J f{x)dn{x) (2.28) 



be a positive moment functional and (p&(x))fcL the corresponding family of monic 
orthogonal polynomials. We may then introduce the Christoffel-Darboux kernel 

K n, ^ _^ Pk(x)p k (y) _ 1 Pn{x)Pn-l(y) ~ Pn-l{x)p n {y) , . 

In |R2j . we studied more general multivariable kernels 

For present purposes, the crucial fact is that the following three expressions define 
the same kernel. As is explained in R2J, this can be deduced from results of 
Ishikawa and Wakayama ji W2j . Lascoux |L1| IL2j . Okada [U] and Strahov and 
Fyodorov |SFj . A self-contained proof is given in [R2J. 



Lemma 2.6. // 



K n, \_ 1 deti<i,j<2m(Pn-m+j-l(^i)) / 9 o n \ 

A ™ w ~n m lb II 2 TT (x — x-) ' { } 



then, for any choice of square roots JHcl, 



lll<i<j<2mW X j V x i) l<*,i<2m 



and, for any choice of & such that & + ^ = + 2, 

T-r2m /-m-1 

= „ llj=1 % _ pfaff ((£,• • Ci)l<ix, ..r/)) . 

lll<i<j<2mW W l<«J<2m 
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We note in passing the less symmetric identities 

det 1 < iJ < rn (K n (x i ,y j )) 



ll<i<j<m 

1 



Ui<i<i< m ( x 3 - x i)(yj - yd 



Ul<i<j< m ( x 3 - x i){Vi-Vi) 

m 



X E Yl^Tl^ , M (Pki( X j)) ,M (PkAVj))- ( 2 - 31 ) 

^ Dt l<«,j<m l<«,j<m 

0<fc m <---<fci<n-l i=l - - - - 

Thus, we have two determinant and two pfaffian formulas for the multiple sum in 
([2.31)1 : these give four different multivariable extensions of the Christoffel-Darboux 
formula 

We will apply Lemma 12.61 in slightly modified form. To this end, assume that 
all odd moments of the functional ()2.28|) vanish. Let (q^'kLo an d (r k )^L be the 
monic orthogonal polynomials associated with the moment functionals 

./' ^ J f(-x 2 )dfx(x), f ^ J f(- x2 ) x2 d K x )i 

respectively; the minus signs will be convenient. Then, 

P2k(x) = (-l) k q k (-x 2 ), p 2 k+i(x) = {-l) k xr k (-x 2 ). 

Later, we will choose p k = c k as the continuous g-ultraspherical polynomials in 
(j2.1(J|) . Then, one may use (|2.16|) to identify q k and r k with continuous g-Jacobi 
polynomials. 
Let 

*"<*.»>= £ P 4^- (^32) 

0<fc<n-l "^ fc " 
k=n—l mod 2 

Then, applying (|2.29|) to the systems (qk)' k Lo and (rfc)^ gives 

f>n ( \ _ 1 Pn+l(x)Pn-l(y) ~ Pn+l(x)Pn-l(y) fr> oo\ 

1 ,y) ~ Ik-ill 2 * 2 -y 2 ' { } 

We note in passing that ()2.32|) implies 

K n (x,y) = K n (x,y)-K n ~ l (x,y), 
which, upon iteration, yields 

n-1 

K n (x,y) = £(-l)^ +1 tf' +1 (x,y). (2.34) 

More generally, let us temporarily write and for the kernels obtained 
by replacing (pfc)^ =0 in with (gfc)fc^o an d ( r fc)fclo' respectively. Moreover, let 

K m ■ ■ ■ i %2m) Qmi X li ■ ■ ■ i X 2m)i 
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K m (Xi, . . . , Xira) X\ . . . X2mR m { X-y, ■ ■ ■ > x 2m)- 

Then, applying Lemma [2. 61 to the kernels Q and R, the result may be expressed 
in unified form as follows. 

Corollary 2.7. Under the assumption of vanishing odd moments, the following 
identities hold: 

X n (x) — - deti<j ) j<2 m (Pn-2m+2j-l(Xi)) 

rilil lbn+l-2i|| 2 Ill<i < j< m ( X 'j - X i) 

= ^ ^ r pfaff ({Wj - Wi)K n {x h Xj )) (2.35) 

n 2m f™- 1 / ~ \ 

= FT fc _^ P faff (0-0)/v"(.r,.r ; ) . (2.36) 

Hl<i<j<2mVSj S«J l<i,j<2m v ' 

where wf = —x\ and & + ^j^ 1 = 2 — , 1 < i < 2m. 

The right-hand side of ()2.30|) is an analogue of a rectangular Schur polynomial; 
in fact, it can be identified with a Schur function over an abstract alphabet |Llj . 
These polynomials make sense also for odd-dimensional matrices. Let 

p / x ) — ^^-^-ra^+j" 1 ^)) — d e ti<»,j<m(Pn+j-i(xj)) ^2 37) 

""' '"' ^l<i,j<m{Pj-l{ x i)) Ul<i<j<m( X j ~ X i) 

so that 

Km( X ) = rfiri n ij^ ^ (n-m) 2m ( x ) ■ 

Under the assumption of vanishing odd moments, we will also write 

p („ „ \ _ det l<i,j<m(P«+2j-2(^)) , 9 „„, 

lll<i<j<mV x j "''i/ 1 

so that 

Km(. X ) = Tfrn — il [To ^(n+l-2m) 2 ™ (^) • 

lli=l HPn+l-2i|| 

Then, if Q and R denote the kernels obtained by replacing p^ in ()2.37|) with 
and rfc, respectively, we have 

P (2 „)-(xi,...,x m ) = (-l) mn Q nm (-x 2 ,...,-x 2 „), 

^(2n+l) m (^1) • • • > x m) = ( — l) mn Xi ■ ■ ■ X m R„m (— x\, . . . , — X^J . 

It will be useful to note that 

lim t" n P n m+i(xi, ...,x m ,t) = P n m{xi, . . .,x m ); (2.39) 

this is easily derived from (|2.38jl . 
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Finally, we mention the integral formula 

P„m [X\, . . . , X m ) 

= n!rr 1 ,, |, 2 / n fa - y ^ II ~ y ^ 2 ^(yi) • • • MVn), 

• 1 Li = l UPl— 1 II ^ 1 <-„-<"_, l^i^o^n 



l<j<m l<i<j'<n 
Kfc<n 



which is at least implicitly due to Christoffel, see also [BHl IR2j . Applying this to 
the polynomials q^ and gives 

I"(2n) m { x li • • • > x m) 



n] - nr=i 

/ II II (y"-yi) 2 dfi(yi)---d^y n ), (2.40a) 



l<j'<m l<i<j<n 
Kfc<n 



P(2n+l) m (^1 , • • • , 2? m ) — , n « M 



i-1 



2 



X 



n^ 2 -^) n (^-«?) a ii^w- d ^)- ( 2 - 4ob ) 



l<j<m l<i<j'<n fc=l 

l<fc<n 

These formulas will be used in the Appendix. 

3. Relation to Kawanaka's identity 
Kawanaka [Kj Theorem 1.1] obtained the Schur polynomial identity 

At a€/j * i=l v l<i<j<m 1 3 

Here, the sum is over all partitions H\ > /x 2 > • • • > > and denotes the 
hook-length at the box a of the diagram of /i, see |Maj . We will recover a proof of 
(j3.1|) below, see Remark [5.21 

Although it is not mentioned by Kawanaka, (|3.1|) can be viewed as evaluating 
a Schur Q-function at an infinite geometric progression. To see this we observe 
that, writing fcj = + m — i, the determinant in (|2.7|) equals 

J" J ip^i %j ) ■ ■ ■ i -Em) ■ 

l<i<j<m 

Moreover, it is easy to check that (cf. [Ma, Examples 1. 1.1 and 1.1.3]) 

n' a (-g; gk tt g fcj - g fct = tt 1 + Q h(a) 
(q;q)k- q k J + q ^ 11 1 _ «/»(<») ' 

i=l l<i<j< m ^ ^ aSAt ^ 
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m ' -jU+2—i. „\ i „c(a)+m— n~ 1 



M (-g" n ; gk y ' M (-q n+2 ~ l ; q)^ 11 1 + q c 



i (-g _n ; ?)* 1 (-g n+2 -*; ?)<-i ^ l + <f 

where c denotes content. Plugging all this into ()2.7|) gives 

^ ^)=n^ n 



n+l— i 



l<i<j'<m 



y IT \n n, (3-2) 

0</^ m <---</^i<n+l— m aG/i ' ' 



Let us now take the limit n — > oo, which can be justified analytically if |g| < 1 
and < 1 for all i. Note that 



■y ^c(a)+m— n— 1 

lim 1 I — 



1 _|_ c(a;+m— n—i 



_|_ qc{a)+m— n— 1 

which may be absorbed into s M by homogeneity. Thus, we obtain 



h{a) 



\im o p n (x 1 ,...,x m )= ( n n f~few (3 - 3) 

where the sum is computed by (j3.1j) . Although the corresponding Schur Q-function 
identity is equivalent to (j3.1j) . it seems not to have appeared explicitly in the 
literature. 

Proposition 3.1. For A a partition of length m, 

i=l W'^ A » l<i<j<m 1 9 

where the left-hand side is interpreted as 

lim Q x (l,q,...,q n ), \q\ < 1. 

n— >oo 

In terms of tableaux, the result may be written as follows. 
Corollary 3.2. For A a strict partition of length m, 



2^ 11 ( g;g ) A _ 11 l_ g A i+ V 191 < X ' 

■ =1 ^'^ A * l<i<J<m y 



where the sum is over all marked shifted tableaux of shape S(X). 
This should be compared with the identity 

m i A A 

i>' TI -n^ n <«) 

T i=l ^' ^ /Al l<i<j<m * 
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where the sum is over column-strict shifted tableaux of shape S(X), that is, only 
unmarked symbols appear, so that conditions (l)-(3) in Section [2~T1 mav be sum- 
marized as 

(4) The labels increase weakly along rows and strictly down columns. 

As was pointed out by Stembridge |Stej . ()3.4|) is equivalent to a result conjec- 
tured by Stanley jSij and first proved by Gansner [0]. Comparison with Corol- 
lary 13.21 suggests the following problem. 

Problem 3.3. Prove directly (that is, without summing the series) that 



E i m -U(- 1 -^. E 



,\t\ 

T marked i=l T column-strict 

where A is a strict partition of length m and the sums are over shifted tableaux of 
shape 5(A). For instance, writing and Ck for the set of marked, respectively 
column-strict, shifted tableaux T of shape S(X) such that |T| = k, construct a 
map : M k — > Uj<fc Q such that 

rn 

£ \M k+j n <p-\x)\q k = fj(-l; q) Xi , x G C y 

k i=l 

This would lead to a new proof of the Stanley-Gansner identity as a consequence 
of Kawanaka's identity (and vice versa). 

4. Relation to a discrete Selberg integral 

Krattenthaler \Ki\ Theorem 6] gave the following multiple extension of the q- 
Chu-Vandermonde summation, which was applied to enumeration problems for 
perfect matchings: 

e n ^-^fi tttfy ' 

0<fe m <---<Ki<ri l<i<]<m i=l t\ + < i 

An equivalent identity was previously obtained by Milne |M1| Theorem 5.3]. More- 
over, as noted in |Krj . another equivalent formulation is a special case of a discrete 
Selberg integral conjectured by Askey jX] and proved by Evans jEj. However, the 
form given by Krattenthaler is more useful for our purposes. 

In the special case x = y = —q, (|4.1|) evaluates the Schur Q-polynomial 
Q\{1, <?,•••, q n ), where A is the odd staircase partition (2m— 1, 2m— 3, . . . , 3, 1). To 
see this, we let Xj = g 2 '" 1 in (|2.7|) . The resulting sum contains the Vandermonde 
determinant 

m 

det (q<? i - r > k ')=f\q 2k * TT {q 2k * - q 2k *) 

l<i,j<m '1.1. J. J. 

i=l i<*<i<™' 
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and is thus evaluated by ()4.1|) as 

PJq,,/,...,,/"'- 1 ) 



)<k m <-<ki<n l<i<j<m i=l HJK% W ' yyn 

-n(™)aK 2 3 m ) fr (-g»-g»g;g)*-i(g i+m ;g)"+i-"> f4 2 ) 

' 7 11 (?;?)r — 



-l)(?VCtT)j] 



■ _ I ! ijn+i—m 

m 



rj (?;? 2 )i-i(?; <? 2 V 

To check the equality of the last two expressions, the elementary identity 

m m 

J]W; q) m = JJiaq*] q) l {aq i ; q)^ u (4.3) 



i=l i=l 



with a = 1, is useful. 

Since, by anti-symmetry, 

P n (q, g 3 , . . . , g 2 " 1 " 1 ) = {-lp) P n {q 2m ~\ ■ ■ • , Q 3 , ?), 

we obtain the following Schur Q-polynomial identity. Although it is equivalent to 
a special case of (|4.1j) . we have not found it explicitly in the literature. 

Proposition 4.1. When A = (2m — 1, 2m — 3, . . . , 3, 1), t/ien 

(g n+1 " m+i ;g) m 



Q A (l,g,...,g") = 2"V( 3 m )J] 



f = i (<?; ? 2 )i-i(?; 9 2 )/ 

We will recover Proposition 14.11 twice below, see Remark 16.31 and Remark 17.41 
A very similar result holds for the even staircase, see Corollary 17.61 

We write down the limit case q — > 1 explicitly, starting from ()4.2|) with n 
replaced by n — 1. 

Corollary 4.2. When A = (2m - 1, 2m - 3, . . . , 3, 1), then Q x (l n ), that is, the 
number of marked shifted tableaux of shape S(X), with labels from the finite alphabet 
1' < 1 < 2' < 2 < • ■ • < n' < n, equals 

^ TT (n + »-!)!(»-!)! 

(n + z — m — l)!(z + m — 1)! 

i=l v ' v ' 

Problem 4.3. Give a combinatorial proof of Corollary 14. 2\ or more generally of 
Proposition 14.11 
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5. Relation to Christoffel-Darboux kernels 

The following fundamental result identifies P n (x,y) with a Christoffel-Darboux 
kernel for continuous g-Jacobi polynomials. 

Lemma 5.1. One has 

D , v i - q n+1 yPn+i(x)Pn-i(y) - xP n ^(x)P n+1 (y) 

P n [x,y) = — . (5.1) 

v ,yj l + q n+1 l-xy y ' 

Equivalently, fixing square roots \/—x and y/—y and writing £ = a/— x + (y/—x)~ 1 , 

, n— 1 / . \ n— 1 



P„(aM/) = (V=S) (V=j) (v - x) fr&v), (5-2) 
where K n is as in ()2.33|) . witt p n = c n defined in (I2.10|) . 
Proof. By definition, (1 — xy)P n (x,y) equals 

(i - e g !~ g " ( r g ' g " n;g | fc( r g ' g "" ;g N v 

(g;g)n fc 7^ ? +<? w>-9 n ;q)k{q,-q n ;q)i 

'-Q\ q)l ^ q l -q k (-g, q~ n \ q)k-i (-g, g~ n ; q)i-i k , 



E 



(g;g)« ^ q l + q k (g>-g n ;g)fc (g>-g n ;g) 



X {(1 + - ^""-^(l + g')(l - g'-™" 1 ) 

- (1 - q k )(l + g fc - n - x )(l - + q l ~ n ~ 1 )}- 
We observe that the quantity within brackets factors as 

2{q k + q l ){l-q- n - 1 ){l-q k+l - n - 1 ), 
and then split the summand in a different way, writing 

2(q l - q k )(l + q-"- 1 )^ - q k+l - n ~ l ) 
= (l-g fc )(l-g fc - n - 1 )(l + ^)(l + g'- n - 1 )-(l + g fc )(l + g fe -"" 1 )(l-g i )(l-g i -"- 1 ). 
This gives 

(1 - xy)P n (x,y) 

= (-g; q)l ( (-g; q)k~i(q' n ;q)k xk (-q,q~ n ~ 1 ;g)i t 

(-g, g""" 1 ; g)fc x fc (-g; g)z-i(g~ w ; g)i j 
^ (g, -g-"- 1 ; g) fc ^ (q;q)i-i(-q- n ;q)i v 
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Replacing k by k + 1 in the first term and I by I + 1 in the second term yields (|5.1j) . 
It is then straight-forward to derive (J5.2j) . using (J2.ll)) . (|2.12|) and the elementary 
identity 



£ 2 - V 2 = —{x -y)(l- xy). 
xy 



Remark 5.2. By the g-binomial theorem (GR, (II. 3)], 

lim P n {x) = { ; q]qU f { ^4±{-xf = tM^ll 
for |g|, \x\ < 1. Lemma f5. II then gives 

-q;q)lo {q x ,qy\q)°° i/-^ 



□ 



lim P n (x,y) 



(q;q)lo (-x,-y;q)ooi-xy 

By dQ7|) and ()2~27)j) . it follows that 

i- o / \ ( — q\ q)oo tt (q x i'iq)oo tt xj — Xi / co \ 

hm P n {x 1 ,...,x m ) = ^— — II 7 r— }} - 1 (5.3) 

for m even. By (12.8)1 . this holds also for odd m. Comparing ()5.3|) and (J3.3)l yields 
Kawanaka's identity 1)3.1)1 . which thus follows from Lemma 15. II In fact, our proof 
of Lemma 15. II generalizes the proof of Kawanaka's identity given by Ishikawa and 
Wakayama )IW2j . 



We rewrite ()2.34)1 in terms of the polynomials P n , assuming for convenience that 
we have chosen yfx and y/y so that \fxj -Jy = \J—xj\J—y. The resulting identity 
will be used in the proof of Theorem 16.11 



Corollary 5.3. One has 

P n (x,y) = ^^ ^(^"-^(VyPj+^Pjiy) - y/ZPjWPi+M). 



n-l 

+ \fxy 
v w j=0 

Next, combining ()5.2)) and ()2.27|) gives 

2m 

P n (x u . . . , x 2m ) = YliV^) 71 ^ pfaff ((xj - Xi)K n (^ £,-)), 

. . Ki,j<2m 
1=1 — J — 

where £j = a/— Xi + (\/— a^)" 1 - We see that the pfaman is of the form ()2.36j) . with 
Q = Xi, Zi = £j. This shows the following result when m is even. The case off odd 
m follows as a limit case, using (J2.8)l and (|2.39jl . and the expression ()2.12)1 for the 
norms. 
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Theorem 5.4. For any fixed choice of square roots y/—xi, 



m 

rn 



i=l * ' l<t<j<m 



= 1 (?;9)n+l-/ 



where £j = \f—xl + (V — x «) S an <^ where P is defined by choosing pk = Ck in 
(|2.38|) . When m is even, we equivalently have 

2m 

P n (x u . . . , X 2m ) = J](V=^r +1 - 2m J] (X, - Xi ) KZ(Zl, • • • , 6m), 
i=l l<i<j<2m 

where K is defined by choosing p^ = c^ in Corollary 12.71 

We will give an alternative proof of Theorem 15.41 in the Appendix. 

Note that £j is invariant under the transformations y/—xi i— > (y^—x^ 1 or, equi- 
valently, J~xi i — ► — (y 7 ^) -1 . This yields the following hyperoctahedral symmetry, 
which is quite non-obvious from the definition (|2.6j) . 

Corollary 5.5. The function 

-1 Pni^li • • • i -Era) 



,m-n- 

.1 ; 



, 1 ni<i<j<m( X i X «) 

is invariant under the action of the hyperoctahedral group B m generated by per- 
mutation of the variables together with inversions ^fxl \— > — (y^) -1 . 

It should be noted that, although phrased in terms of square roots, this is a 
polynomial statement. For instance, the symmetry under ^px~\ \— > — (y^) -1 can 
be rephrased as 

m m 

- XiXi) P n (xi,X 2 , • • • , X m ) = {~Xi) n \\{Xi - Xi) PniXi 1 , X 2 , • • • , X m ). 
i=2 i=2 

We note in passing the formulas for P n obtained from (|2.31|) . These will not be 
used in the remainder of this paper. 

Corollary 5.6. One has 

P(x , x m y ) - UZ =1 (y-^) ( P n (Xi,Vj) 



ki,...,km=n— 1 mod 2 

x det (^ 1 - fc ^ 2 P fei (x,)) i det (yj^-^C^)) 
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Turning to (|2.H5|) . we have 



Ul<i<j<2m( X j ~ X i) ( a f W 3 ~ W 



P n {.Xi, . . . , X 2m ) — — x " - pfaff I — —PniXijXjjj. 

Ui=l *i lilKi^J™! ~ W i) l<i,j<2m V Xj ~ Xi 

where W{ = ^/xl — (y^) -1 , so that 

Wj -Wi= 1 {^fx] - y/Xi)(l + y/XiXj). 



This gives the following pfaffian formula, which forms the basis for Theorem 16.11 
Corollary 5.7. For m even, 



P n (x 1} . . . , x m ) = Yl , ! , pfaff 



~t~ v/ Xj f 1 ~\~ y/X^Xj 

1 n \ Xi . X 7 



l<i<jr'<m 



-|- v/ X{Xj l<i,j<m \ \jx~i ~\- v/ 



Rewriting the determinant formula ()2.38|) in terms of the one-variable poly- 
nomials -P n (x) and reversing the order of the columns, we obtain the following 
identity. 

Corollary 5.8. One has 

J J (1 SjXjf) P n {x\ , . . . , x m ) 

l<i<j'<m 

Corollary 15 . 81 seems interesting enough to write down also in standard notation 
for Schur Q-polynomials. 

Corollary 5.9. For A a partition of length m, 

(g n+l-m+f . q) ,_ i ! 

x det (^-^(l, g, . . . , <f +-+^)) . 

Replacing n by n — 1 and letting q — > 1 gives the following simple determinant 
formula for 

Corollary 5.10. For A a partition of length m, 

m 

11 ( n _ m + , _ i)! ^±± ^ Aj + A, 

where fk are the hypergeometric polynomials defined in ()2.23|) . 



g A (l,g,...,g n )-n ( +1 . m+ ' ^ II f-^ 

"- 1 v H ' H,% 1 Ki<i<ra ^ 
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For future refrence [R3j . we rewrite Corollary 15. 101 in terms of generalized Schur 
polynomials and multivariable Christoffel-Darboux kernels corresponding to the 
continuous dual Halm polynomials and p^ introduced in (|2.21|) . Let P„2 be 
the polynomial obtained by choosing p k = p^ in ()2.37|) . and let K"^f be the kernel 
similarly obtained from Lemma \2. 01 Using the expression (|2.22jl for the norms, we 
arrive at the following result, which can also be obtained directly from Theorem 
15.41 as a limit case. 

Corollary 5.11. Let n — m = 2k + e, with k an integer and e G {0, 1}. Then, 



Q(Ai,...,A m )(l n ) 



<2^m(2n+l— m) / -^\km 



U X II (A*-A,)Pg(-Af. 

i=l l<i<j<m 



m 



[n — i) 



— A m )- 



In particular, 



<5(Ai,...,A 2m )(l 2n ) = 4 m Yl (At - Aj) K%°(-\1, . . . , -\l m ), 

l<i<j<2m 



2m 

Q(A 1 ,...,A 2m) (l 2n+1 ) = 4 m J] A, n (Aj - A,) ^(-A?, . . . , -AD- 

i=l l<i<j<2m 



6. A PFAFFIAN HYPERGEOMETRIC IDENTITY 

Using the pfaffian formula of Corollary 15.71 we derive the following hyper- 
geometric identity. Note that it displays the hyperoctahedral symmetry of Corol- 
lary EH 

Theorem 6.1. For any fixed choice of y/xl, 



Pn j • • • jX 



1-q 



l-q 



n+l \m m 

n< 



n \ 



X 



i=l l<i<j<m 

2-^ 11 i _ qki+kj+i 11 



fci,...,fc m =0 l<i<j<m 



i=l 



(q,q,q 3/2 ,-q 3/2 ]q)ki 



q k \ (0.1) 
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Proof. We first consider the case m = 2. Using (|2.17jl in Corollarv l5.Hl and changing 
the order of summation gives 

P n{X, y) = JZ ^ {y/Xy) 



(1 — q) 2 1 + Jxy 



n-l 

E (W +1 )(W +2 )g^ 

j'=max(fc— 1,2 — 1) 

x {(ff-^.^jg)*^,^ 3 ^), - (q~ j ,q j+2 ;q) k (q- j ~\q j+3 ;q)i}. (6.2) 



Let 5" denote the inner sum in ()6.2|) . It can be rewritten as 



n-l 

5 = e (_i)^ g ©+a)-(^)(i+i)-i (? i- fc+ 2. g)2fe(? i-^. g)2z 

j'=max(fc— 1,2— 1) 

X {q l (l - q^ k+2 )(l - qi- l+1 ) - q k (l - q^ k+1 )(l - q> +l+2 )}, 
where the expression within brackets factors as 

0?'-<7 fc )(l-^' +3 ). 
Replacing j by n — 1 — j, elementary manipulations give 

S = q^(q l - q k ){l - g 2 " +1 )(g-, q n+1 ; q) k (q' n , <f + 1 ; q\ 

min (n-fc,«-2) 2n-l+2j (Jk-n „l-n. _\ 

l_ 9 -an-i (q- k ~ n ,q- l - n ;q)j 



Here, the sum is a very-well-poised 60s, which by [GRJ Eq. (11.21)] equals 

(l-g 2 "+ 1 )(l-g fe +'+ 1 ) ' 

so that 
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This gives 
P n (x,y) = qi n — (v 7 ^) 



1 — q ) 1 + -v/xl/ 



k,l=0 



q i- q k (q n ,q n+2 ,^/q/x,-^qx;q) k k 
X 2_. [l-qk+l+l (g,g,g3/2 ; _ g 3/ 2 . g)fc 



{q r \q n+2 ,^qjy,-^qy\q)i A _* 

X 7 372 372-^ 9 ' 6 ' 3 

{q, q, q 6 '\ -q 6 l l \ q) l J 

which is the case m = 2 of (|6.1j) . 

Suppose now that m is even. We plug ()6.3|) into Corollary 15.71 thus expressing 
pfaffian of two-dimensional sums. Applying Corollary 12 . 31 leads 
to an m-dimensional sum, each term containing a pfaffian of the form (|2.26j) . This 
proves (jbMj) in the case when m is even. 

Finally, if m is odd, we use ()2.8j) to write 



p ( \_v (?;?)n+l (-I)"" p / ,n 

J ni^lj ■ ■ ■ , X m J — U m 7 s *n+l\%l, • • • j ^m, EJ- 

(-9! 9)n+l Xf-X m 

Applying to the right-hand side, only the term with k m+ i — n + 1 survives 
in the limit. After simplification, one arrives at the desired expression. □ 

In contrast to (|2.fi|) . (jfj.ljl is regular in the limit 

g A (l" +1 ) = 2 m lim P„(g Al , . . . , g Am ). (6.4) 
g-+i 

Corollary 6.2. Let X be a partition of length m. Then, 

, n+ h_,„ , „™ tt h-kj j^(-7i,n + 2,(l-Ai)/2) fc( 



9»(i-)= ( 2 B+ 2)- £ n mhn 



fci + A;,- + 1 11 (1.1.3/2) fc . 

fel,...,fc m =0 l<i<j<m ^ i=l \ 1 1 I 

Remark 6.3. If Xi = q 2t ~ x ■, corresponding to the odd staircase partition, the sum 
in ()6.1|) reduces to the term with fcj = % — 1. After simplification, one recovers 
Proposition 14.11 



7. Determinantal hypergeometric identities 

By inserting hypergeometric expressions for P n (x) into the determinant in Corol- 
lary EH1 one may obtain further formulas for P n (xi, . . . , x m ). We are particularly 
interested in cases when the result simplifies using a determinant evaluation such 
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as (SI Lemma A.l] 

nAX^AC/X^A = (?) (?) 

iSd^KiBX^BC/Xi-q)^) q { ] 

* TT (x, - - x^/c) ft ^ ^g M ; g)t-i (71) 



l<i<j<m 1=1 * 6 3 ^.j III -L 

or the degenerate case 

det ((AX i ;q) J „ 1 (BX t ;q) m _ J ) 

X<i ,j<m 

m 

= q^A& J] (Xi-X^Jl^B/A^i-L (7.2) 

l<i<j'<m i=l 

Multiple hypergeometric sums obtained in this way have been called "Schlosser- 
type", see [UKllS]. 

First of all, using our original definition of P n (x), we find the following identity, 
which has a very similar structure to sums found in |RSj . 

Theorem 7.1. One has 

X<i<j<m 

= (zMg fr (q n+1 - m+l ; g)i-i 

n+m—1 / 

x ( II (g*'- 9*0(1 -g fc<+fe ' +1 " m " n ) 

fcl,...,fc m =0 \l<i<j'<m 

m /"„1— m— n „2—m. „\ \ 

Proof. By f!2.9|) . the right-hand side of (|5.4jl equals 

TT {-q,q)n+i-i \- , v TT IZM .gjfci fei+ g (0-i 

Replacing fcj by er(z) and interchanging summations gives after simplification 



. n (?) (-g;g)^+n-i fT (g" +1 - m+ ';g) 



i-1 

(g;g)™ +n _i J* (_gn+i-m+i. ^ 



V det ( (g »9 ' gJj-i \Tj (-g»g ^)kj x kj 

^ _ n x<ij<m V(-g^ +1 - m - n ,-g- fc sg)j-i/ fj (q, -q l - m ~ n ] q) ki 
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By (17. lj) . the determinant equals 

m 



(-l,-g 1+T "- n - 2i ;g)i 
y j y j qjm—i 



After further simplification, one arrives at the right-hand side of (|7.3|) . □ 
Remark 7.2. Writing Theorem 17. II as 

n+m— 1 m 

IT ^ -^j)^nOri,---,£m) = • • • ,^m) 

l<i<j<m ki,...,k m =0 i=l 

one easily checks that (— l)' fc 'x(/c) is antisymmetric and invariant under the reflec- 
tions ki i— > n + m — 1 — fcj. This fact is equivalent to Corollary 15.51 In analogy 
with the Schur polynomial expansion (|3.2|) . one may exploit this symmetry to 
rewrite Theorem 17.11 in terms of characters of the classical groups SO(2m) and 
SO (2m + 1), according to whether n + m is odd or even, respectively. 

Inserting different expressions for P n {x) in Corollary 15. 8[ one may obtain further 
hypergeometric formulas. We will not exploit all possibilities, but merely give a 
few examples, corresponding to the eight expressions in (J2.2(J|) . The resulting 
identities share the nice properties of Theorem that is, they are regular in the 
limit q — > 1, have completely factored terms, display hyperoctahedral symmetry, 
and have nice specializations at staircase-type partitions. 

For instance, assuming that n + m is odd, we insert ()2.20a|) into (|5.4j) . After 
simplification, we obtain 

[ [ (1 •Ei-Kj)Pri{p^li • • • i •^m) 
l<i<j'<m 

n —\nm m ( n n+l-m+i. _\ 

Q 2 77 {Q ijLH x (n+m-l)/2 



n 



(1 - q) m A jL (_gn+l-m+i. g) i _ 1 (g(l-«-m)/2 ; _g(3+n-m)/2. 

(n+rn-l)/2 rn , (1 _ n _ m )/ 2 _„(3+n-m)/2 „/ r .-„}, 

x TT W_ ; g_ , g/Xj, gjfc. fe . 

fel ,.il=o fi (g,g,g 3 / 2 ,-g 3 / 2 ;g) fcl 

x det ((g^ +(1 -^ m)/2 ;g)^ 1 (-g^ +(3+ - m)/2 ;g) m _,) 

l<ij'<m 

where, by (|7.2jl . the determinant equals 



m 

1 / m\ n + 1 / m\ 



l<j<j'<m i=l 

Repeating this for all eight expressions ()2.20|) yields the following result. In the 
case of ()7.4c|) and (|7.5dj) we have also used (|4.3j) . 
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Theorem 7.3. If n + m is odd, then rii<j<j<m (1 ~~ ^iXj)P n (xi, . . . , x rn ) is equal 
to each of the four expressions 

(_l)(-),-K^)-f m n __ {q n + l-m + >. q)i Jn+m _ m 



(1 - g)™ 11 (g(l-™-™)/2 ) _g(3+n-m)/2. ? ) 

(ra+m-l)/2 / 

x e ( n (<? fej -g fci ) 

fci ,...,fc m =0 \l<i<jr'<m 



X 



U 1 r 3/2 3 /2 , ^ » 



g 4 " m ( g; g)^ +1 _ m)/2 " (g w+1 m+ '; g)t_i(g 3/2 ; g)( ra+m+ i_2i)/2 ( n +m-i)/2 

(95 ?)(^ +m -l)/2 fj (-g 1/2 ;g)(n+ m+ l-2i)/2 ^ 

(n+m-l)/2 / 

x e n (^ -g fci ) 

fci ,...,fc m =0 \l<i<jr'<m 



X 



™ / (l-n-m)/2 _ fl (3+n-m)/2 1/2 nV^/r.-n), , 
II ~ 7 1/2 3/2 T , (7.4b) 



i=i 



1 2 TT^™+ 1 - m +*. ^ A n + m - 1 )/ 2 



(n+m-l)/2 

x e i n (^-^) 



fci,...,fc m =0 \l<i<j'<m 

( g l-n- m^ g 3+n-m ? ^ g 2^ 

(g,g 2 ,g 2 ,gWk 



x ll- — ,:,;r; v . <™o 



i=i 



-rim 



9 TT/ n+l-m+i. \2 (n+m-l)/2 

(1 _ g)2m(g 3 + n~ m . g 2 ) m_ i llW >^ i 



(n+m- 1)/2 

x e ( n (^-^) 



fci,...,fc m =0 \l<i<j<m 

m ( n l—n—m „3+n-m (7 2„ „2 /„.. _2\ 

ly ; V x »? V / x »> H )ki J2k, 

(q 2 ,q 2 ,q 3 ,q 3 ;q 2 )k., 



i=l 
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whereas if n + m is even, Yli<i<j< m 0- ~ x i x j)Pn( x i, ■ ■ ■ , x m) equals 



(1 - q) 2m f \ ( g (2-n-m)/2 ; _ g (4+n-m)/2. 

m (ra+m-2)/2 / 

x[i(i-«)r- 2)/2 e n ^-^ 

i=l fci,...,fe m =0 \l<i<j<m 

™_ i (2-n-m)/2 _ fl (4+n-m)/2 n/ r -a)u , \ 

XF , '2 3/2 ./iT ^ ' ^ 

f = | (g,g 2 ,g 3 / 2 ,-g 3 / 2 ;g) fci ' 

r* (w - 1)m (-g;g)fc +2 - m ) /2 -pr (9 w+1 - m+t ;9),-i(g 3/2 ;g)(n +m -2. /2 

(l-g)-(g;g)- +m _ 2)/2 H (-gS/2; g) (n+m _ 2i)/2 

m (n+m-2)/2 / 

xnfl-.)^ 2 E II 

i=l fci,...,fc m =0 \l<i<j<m 

( 5 (2-n-m)/2 > _ g (4+n- m )/2 ; ^1/2^ g l/2/ x , g ) fe 

(g,g 3 / 2 ,g 3 / 2 ,-g;g) 



>< n - — -,?^^-„.„r — « fe • p-*) 



i=l 



„-i(n-l)m m 
I ' JJ (gB+ l^H ;g)3Li 



(l_ g )2 m(g 4 + „- m;g 2 ) m_ 2 ll 

m (n+m-2)/2 / 

xII( 1 -^K (n+m " 2)/2 E II (9^-9**) 

i=l fci,...,fc m =0 \l<i<j'<m 

(g 2 ,g 2 ,g 3 ,g 3 ;g-j;. 



8=1 



— (n— l)m m 

1 Tit n+l-m+i. \ 

(1 _ g)am(1 _ g 2 )m(g 4+n- m . g2) -_ 2 11 ^ , <?V 

m (n+m-2)/2 / 

x nci - ^K (n+m - 2)/2 e n (^ -^; 

i=l fci,...,fc m =0 \l<i<j'<m 



m 



Note that all eight expressions are regular in the limit (|6.4|) . We do not write 
down the resulting identities for Qx(l n ) explicitly. 
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Remark 7.4. By specializing the variables Xi, each of the eight sums in Theo- 
rem [731 m ay be reduced to the term with ki = i — 1. In the case of (J7.4a|) and 
(|7.5a|) . this happens if = q\ leading to the identity 



Q, ■ ■ ■ , Q n ) = 2 m (-l)U Jp n ( g , q \ . . . , g™) 

/ i in m ( n n+l—m+i. „\ 

f = \ (q; q 2 )i 

which follows more easily from |Ma[ Ex. III. 8. 3] 

Q(m,m— 1,...,1) 2 Sj^jTijjTi— ■ 

In the case of f!7.4b|) and f)7.5b|) . one should let Xj = g i_ ^. We write down the 
result in Corollary 17.51 Similarly, when Xi = g 2i_1 , (|7.4cj) and ()7.5c|) reduce to 
Proposition 14. II and when Xi = g 2 *, ()7.4d|) and (|7.5d|) yield Corollary 17.61 

Corollary 7.5. If Xi = q % ~^ , then P n (xi, . . . ,x m ) equals 

(_l)(T)gl(T) fj 3"+!-™+*; g)j_i(g 3 / 2 , -g; g)( w +m+l-2i)/2 

f = { (-9, 9 1/2 , ? 3/2 ! g)<-l(?, -5 1/2 ; <?)(n+m+l-2i)/2 

if n + m is odd and 



1/2 -g)m TT (g, m+L , q)i-l(q 3/2 ; g)(n+m-2i)/2(-gi 9)(n+m+2-2i)/2 

|Ll (-g^ 3/2 )g 3/2 ;g)i-l(-g 3/2 ,g;g)(n.+m-2i)/2 



ifn + m is even. 

Corollary 7.6. When A = (2m, 2m - 2, • • • ,2), £/jen 



g A (l,g,...,g")=2V^ +1 )n (9 , 2 ,2 9)t - 

Appendix. Alternative proof of Theorem 15.41 

The proof of Theorem 15.41 given above used the explicit expression for q- ultra- 
spherical polynomials as basic hypergeometric series. Here, we give a different 
proof using only their orthogonality measure. This might be useful for the purpose 
of generalization. 

Let 

{ ] 2n(-q;q)l(-qe™,-qe-™;q) 00 - 

By [GR, Eq. (7.4.15)], 

f(x)dfx(x)= f (2 cos 0)w (9) d9 (A.l) 
Jo 



" 
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is the unique measure such that the polynomials (j2.1()j) are orthogonal with norm 
(J2.12j) . By (|2.4U|) . Theorem 15.41 is then equivalent to the integral formulas 

p n (x u ...,x m )=-j[ r g '? > n^ n ( x J~ x i) 

i=l j=l l<i<j<m 



I II (x, + x- 1 + e 2 ^ + e" 2 ^ 

Jo<0i,...,Oi<* i<j< m ,l<k<l 



x I I (e 2iek +e- 2 ^-e 2ie i-e- 2 ^) 2 l[w(6 k )d6 kl (A.2a) 

i<j<fc<« fe=i 
when n — m = 2/ — 1 and 

P n (xi,...,x m ) = - JJ ' - JJg-(l-Zi) JJ (Zj-Xi) 
i=l ^' ^ l i=l l<j<j<m 

x / J] ( Xj+X 7i + e 2 ^+e- 2 ^) 

Ja<ei,...,6i<n l<j< m ,l<k<l 

I 

x TJ ( e *«* + e -2i0 h _ e ™ 3 _ e -2ie 3) 2 + e -^y w (e k) d9kj ( A .2b) 

l<j<k<l k=l 

when n — m = 21. 

To prove this we will use the following alternative expression for Q\. An equiv- 
alent identity is due to Nimmo (Nj Eq. (A12)], see also (Mil Ex. III.8.13]. 

Lemma A.l. Let I be the integral part of (n — m)/2. Then, 
Q(\ 1 ,...,x m )(xi,...,x n ) 

nm—l 

ii r^E^wn^n 



•''/' -'" / \ "~ . , X cr ( m +2i— 1) X (T ( m _|_2i) 



/ I XX x ■ — '7' 



in particular, if I is the integral part of (n+ 1 — m)/2, then 

Pn{pC\i • • • j X m ) 

• j=1 w> yyj ff es n+ i t=i t=i T y 
Proof. By definition, 

0772 I 



(n _ m) , A, 11 <*) ^ z^-x^ 

l<j<jr'<n 



We write 



where 
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l<i<m Xa ^ ~ Xu ^ 
l<i<j<n 

A= II _ — — = sgn(a) 1} J , 

l<i<j<n Xa ® X "W l<i<j<n Xi Xj 



"= n 

Next, we note that 



rn+l<i<j<n v 1 yJ ' 



Indeed, if n — m is even this is Schur's pfaffian evaluation ()2.25|) . while if n — m is 
odd it is the case X2 m = 1 of (|2.25J1 . It follows that the factor B can be traded for 

(n — m)\ j-j- X<r(m+2i-l) - X<r(m+2i) _ 



2H\ 



=1 ^o-(m+2i-l) + x a(m+2i) 



since the result is antisymmetrized over S n we need not perform the antisym- 
metrization over the smaller group S n - m . This completes the proof. □ 

It will be convenient to introduce the functional 

r* f(-e 2ie ) - f(-e- 2ie ) 
Jo e — e 

defined on Laurent polynomials. 

Lemma A. 2. One has 



J ^ 

Proof. Since 



t k d\(t) = \, keZ. 

w 1 + q k 



f(t)d\(t) = - / /(r x )dA(0 



we may assume that k is positive. We may then write 

-l) fe+1 f w e 2idk — e~ 2i9k 
~2 J w e™-e 

\k+l k ~ l 



1 \k+l x pTV 

tl_J2 / e 2i9 ^ +1 ^w(6)d6. 

2 3=0 J ~« 
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By Eq. (7.4.18)], 

e 2 ^ w{6) d6 = 2(-l)Y l_ 1 ~ q = 2(-l) fc (a fe _ 1 - a k+1 ), 

(1 + g fe ^(l + q k+l ) 

where a k = q k /(l + q k ). Thus, the above sum telescopes, giving 

t k d\(t) = a_ fc - a k , 

which simplifies to the desired expression. □ 
We now use Lemma [A. 21 to represent the numbers 

I qcr(m+2i)—cr(m+2i—l) 

\ _|_ qu(m J r2i)— <r(m+2i— 1) 

appearing in ()A.3)) as integrals. This gives 

Pn(%l j • • • j -Km) 

= n e n< w_i ri / h)^*^*- 1 ) dx( t ). 

AL j=l {q ' q>j *£S n+1 <=1 l= l J 

Interchanging summation and integration, the sum becomes a Vandermonde 
determinant, for which we introduce the notation 

A (a;) = det(x^ _1 ) = — Xi). 

i<j 

If n — m = 21 — 1, we thus obtain 

P ( \- 1 TT g)j 

Wl ,...,rr m ; 2i/! JUL 

x y A(xi, ...,x m , tjf 1 , ti,..., tf 1 , *j) dA(ti) • • • dA(<i), 

whereas if n — m = 21, 



Pi \- 1 TT (~g? g)i 



x y A(xi, . . . , x m , t x \ ti, . . . , t, \ tj, 1) rfA(ti) • ■ • dA(t { )- 
Next, we apply the following elementary identities. 
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Lemma A. 3. One has 

A(xi, . . . , x m , t\, , . . . , ti, t} ) 

m I 



n^n^- t 7 i ) A ( a; ) A ( r ) 2 n & 

i=l j=l l<j<rn,l<k<l 



A(xi, . . . , x m , ti, t-^ , . . . jtijt^ ,1) 



\{^ - x t ) Hit, - tf)(2 - r,) A(x)A(r) 2 J] & - r,), 

i=l j=l l<j<m,l<k<l 



where & = x { + x i 1 , r i = t i + t i 1 . 
Writing 



(ti ~ tfmrj) d\(tj) = 2 / f(-e 2 ^ - e-**')w(0j) dB jt 

Jo 

we recover (|A.2|) . This completes our alternative proof of Theorem 15.41 
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